{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Getting Started with Intel® Extension for PyTorch (IPEX)\n",
    "This code sample will guide users how to run a PyTorch inference workload on CPU by using oneAPI AI Analytics Toolkit and also analyze the CPU usage via oneDNN verbose logs."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resnet50 Inference on CPU\n",
    "***\n",
    "This section shows users how to run resnet50 inference on CPU."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### prerequisites"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ignore all warning messages\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "import os"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the installation path of your oneAPI AI Analytics toolkit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%env ONEAPI_INSTALL=/opt/intel/oneapi"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download the resnet50 inference sample from Intel® Extension for PyTorch (IPEX) github repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://raw.githubusercontent.com/intel/intel-extension-for-pytorch/main/examples/cpu/inference/python/python-scripts/resnet50_general_inference_script.py"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run resnet50 on CPU"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Run on CPU via Intel® Extension for PyTorch (IPEX)\n",
    "There is a PyTorch conda environment with Intel® Extension for PyTorch (IPEX) installation in current AI Kit installation.\n",
    "Users could run resnet50_general_inference_script.py on Intel CPU on this PyTorch conda environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile run.sh\n",
    "#!/bin/bash\n",
    "source $ONEAPI_INSTALL/setvars.sh --force > /dev/null 2>&1\n",
    "source activate pytorch\n",
    "echo \"########## Executing the run\"\n",
    "DNNL_VERBOSE=1 python resnet50_general_inference_script.py > infer_rn50_cpu.csv\n",
    "echo \"########## Done with the run\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Submitting build.sh and run.sh to the job queue\n",
    "\n",
    "Now we can submit build.sh and run.sh to the job queue.\n",
    "\n",
    "NOTE - it is possible to execute any of the build and run commands in local environments.\n",
    "To enable users to run their scripts either on the Intel DevCloud or in local environments, this and subsequent training checks for the existence of the job submission command qsub. If the check fails, it is assumed that build/run will be local."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! chmod 755 ../../q; chmod 755 run.sh;if [ -x \"$(command -v qsub)\" ];  then  ../../q run.sh; else ./run.sh; fi"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analyze Verbose Logs\n",
    "***\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download profile_utils.py to parse oneDNN verbose logs from previous section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://raw.githubusercontent.com/oneapi-src/oneAPI-samples/master/Libraries/oneDNN/tutorials/profiling/profile_utils.py"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: List out all oneDNN verbose logs\n",
    "users should see the verbose log listed in the table below.\n",
    "\n",
    "|Log File Name | Description |\n",
    "|:-----|:----|\n",
    "|infer_rn50_cpu.csv| log for cpu run |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "filenames= os.listdir (\".\") \n",
    "result = []\n",
    "keyword = \".csv\"\n",
    "for filename in filenames: \n",
    "    #if os.path.isdir(os.path.join(os.path.abspath(\".\"), filename)): \n",
    "    if filename.find(keyword) != -1:\n",
    "        result.append(filename)\n",
    "result.sort()\n",
    "\n",
    "index =0 \n",
    "for folder in result:\n",
    "    print(\" %d : %s \" %(index, folder))\n",
    "    index+=1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2:  Pick a verbose log by putting its index value below\n",
    "Users can pick cpu log for analysis.   \n",
    "Once users finish Step 2 to Step 7 for one log file, they can go back to step 2 and select another log file for analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "FdIndex=0"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Parse verbose log and get the data back\n",
    "> Users will also get a oneDNN.json file with timeline information for oneDNN primitives. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "logfile = result[FdIndex]\n",
    "print(logfile)\n",
    "from profile_utils import oneDNNUtils, oneDNNLog\n",
    "onednn = oneDNNUtils()\n",
    "log1 = oneDNNLog()\n",
    "log1.load_log(logfile)\n",
    "data = log1.data\n",
    "exec_data = log1.exec_data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Time breakdown for exec type\n",
    "The exec type includes exec and create. \n",
    "\n",
    "|exec type | Description |  \n",
    "|:-----|:----|  \n",
    "|exec | Time for primitives exection. Better to spend most of time on primitives execution. |  \n",
    "|create| Time for primitives creation. Primitives creation happens once. Better to spend less time on primitive creation. |  "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5: Time breakdown for architecture type\n",
    "The supported architecture only includes CPU.  \n",
    "so users should see 100% CPU time. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "onednn.breakdown(exec_data,\"arch\",\"time\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 6: Time breakdown for primitives type\n",
    "The primitives type includes convolution, reorder, sum, etc.  \n",
    "For this simple convolution net example, convolution and inner product primitives are expected to spend most of time.  \n",
    "However, the exact time percentage of different primitivies may vary among different architectures.    \n",
    "Users can easily identify top hotpots of primitives executions with this time breakdown.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "onednn.breakdown(exec_data,\"type\",\"time\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 7:  Time breakdown for JIT kernel type\n",
    "oneDNN uses just-in-time compilation (JIT) to generate optimal code for some functions based on input parameters and instruction set supported by the system.   \n",
    "Therefore, users can see different JIT kernel type among different CPU architectures.  \n",
    "For example, users can see avx_core_vnni JIT kernel if the workload uses VNNI instruction on Cascake Lake platform.  \n",
    "Moreover, users can identify the top hotspots of JIT kernel executions with this time breakdown.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "onednn.breakdown(exec_data,\"jit\",\"time\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output(both stdout and stderr) is displayed on the command line console"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('[CODE_SAMPLE_COMPLETED_SUCCESFULLY]')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py39",
   "language": "python",
   "name": "py39"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
